Version management in Data Manager

During first-time onboarding, you provide a certain set of accounts, regions, and datasets to Data Manager. Based on what you provide, templates get generated and deployed in your environment.

Eventually your environment changes and you edit the data input, such as dataset or account list or region. When this gets persisted to the KV store, the data input in Data Manager is mismatched with the previous deployment in the AWS accounts. The mismatch exists because you need to deploy new templates based on the new data input.

CloudFormation StackSets are created with tags. These tags propagate to every stack instance in every region, and also to every resource created by every stack instance. The tag indicates the version of the Data Manager data input. The version persists in the KV store as the data input changes. The AWS tag name is SplunkDMVersion, its value begins with 1 and monotonically increases.

The version is incremented only when the dataset data input changes because this is the only condition when the template itself changes. Changes in accounts and regions don't affect the template, so also don't impact the version.

For example, consider a first time onboarding. You create a new data input with two areas, two regions, and two datasets. The data input gets persisted with version 1. If you edit the config to remove a dataset, the version increases to 2. If you edit the config to add two new datasets, the version increases to 3.

Data Manager tracks the state of the StackSets and the individual stacks for accounts and regions where stack instances exist. It compares the version on the stack against the data input in the KV store, and it reports the status.